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Leslie D. McLean 
Ontario Institute for Studies in Education 

and 

Anne Keeton 
Queen' s University 



Repeating observations on thB same subject has always been a boon to 
experimenters and a headache to statisticians * Sometiinea measures 
repsated over time are the essence of the study, as in learning and 
developmental , studies , but just as often eKp^srimenters feel that 
sevaral observations^ of the same person are. meaningful or preferable, 
as in using the subject as his own control. Pretest-posttest seeiiis 
the rule rather than the exception* Whether two or twenty, repeated 
observations lack the characteristic, statistical independence ^ 
crucial to every inferential procedure save those which concentrate 
on the correlations themselves* 

Neophytes and Fisherphobes who run the nou'-parainetric domain fiiiJ that 
independence rules that kingdom no less sternly, perhaps more so for 
lack of any rivals* Jlnd these soldiers of the king prove not so strong* 

We grant the strength and clear result where independence prevails, 
especially where backed by randomization* Every raw recruit in the 
statistician arny can be a general with that weapon in his arsenal* 
But too few experimenters are willing to pay the price to achieve 
randomization^ that is, careful advance planning and extensive talking 
to persuade people that the value is worth the effort. In the face of 
repeated me asures, how can we best manoeuvre when complete victory is 
beyond our grasp? Do we attack, dig in=--or retreat? 

In this paper we describe a double frontal attack on some particular 
repeated measures data, an attack which caii be mounted whenev^^r the 
measures have a factorial structure, i.e*, wherever they have been 



riper presented at the annual meeting of the Mierican Educational Research 
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gathered according to soma factorial desigh. The mathod is applicable, 
hOTevar, whorever by design or by theory it is plausible to argue that 
the data have some structure eKpressible in well behaved mathematical 

The concept is by no means new, but practical oxecution requires 
sophisticated computer programs mnd we lack experience in the inter-^ 
pretation of the results, a sophisticated computer program is at hand 
in Multivariance (version 4-1, Finn, 1971) and a full elaboration of 
the method will soon be published (Bock in press) , We present an appli^ 
cation of the method that reveals some of its power and some of its 
challenges. 



h Simple Example 



Because the example to be discussed is somewhat messy, we begin with a 
simple example v/hich illustrates the essence Of the attack, namely 
transformation of the repeated (non-independent) measures into new 
meariingful, orthogonal variables before atteinpting statistical inference, 

Suppose one tests a group of boys and a group of girls on two geometry 
problems. The solutions to the geometry probleiiB yield two repeated 
measures on the two independent groups. There is a design on the ^ 
measures (simple one-way design^ problems A and B) and also on the 
subjects (also one-way^ boys vs girls)* 



Sufficient statistics for the usual normal-tiieory analysis are the four 
means for pr^lem^ and sexes plus the variances anrt covariances (or sums 
of squares and cross products, of course). Look at the four means 
first: . 



Means 

Problem A Problem B 



Boys 
Girls 



11 



'21 



12 



22 



Though there are four cells (4n observations if n boys and A girls) 
tiiere are not 4n independent degrees of freedom* 



3 



The analysis is simplified if the matrix of nieans is transformed by 
a simple 2x2 matrix 



-1 



J ■ 

which convert's the two repeated measures into two other "measures" , 
namely their sum and difference* We now have 



Sum 



Difference 



Boys 



Y +Y 

11 "12 



Y- -Y- 
11 12 



Girls 



^21+^22 



Y --Y 
21 "22 



It is easy to verify that whatever the correlation between h and the 
sum and difference are uncorrelated. Furthermore, these two new 
dependent variables (suni and difference) are themselves .meaningful. 
Separate univariate analysis; of variance would be informative , but a 
pooled analysis is more powerful^, as shown in table 1, 



Table 1 

Interpretation of the Univariate Analysis of 
Variance of Sum and Dif J^rence Scores 



Source 



Mean 



d£ 



neaning of F-^test 



1 geometry Problems 



Sum 

Difference 



Teets whetiier grand mean is zero. Not interesting. 
Geometry problem main effect i mean of differences 



Groups (boys -girls) 1 Sex k Problem Interactions 



Sum 

Difference 



Tests sex main effect 

Bbk by Problem interaction i difference of differenees 



Error 



n^+n^-S Pooled estimate of error 



The essential feature of this technique is the use of an orthogonal 
transformation to convert correlated measures to mprG tractable ones 
before statistical analysis. With more than two measures the new 
transformed variables are not usually uncorrelated but the main pro- 
cadures are identical. It is necessary to do a multivariate analysis 
of the transformed measures ^ taking account of the entire covarianee 
structure, not just the variances. . Where there is also a design on - 
the subjects/ the type of statistical tests one can make depends upon 
the extent of equivalence in the covarianee structure, i.e., the multi^ 
variate analog of homogeneity of variance. This first illustration is 
too simple to reveal the essence of the method/ but it does show how 
the main effects of the repeated tneasures appear in the first half of 
the table and the interactions with the design on the subjects appear 
in the second half* h "bbk effect" in the difference scores is eKactly 
what one means by a "sex^problem interaction". 



The Main E^Miple 

The data tliat serve as the basis for our aKample were obtained in a study 
conducted by Keeton (1973). The design is described in Table 2. 
Isqploratory data analysis led the autiior to create a new variable, 
Scoring, namely the number of digits recalled correctly from the first 
half of the series (primacy) and from the second half (recency) . 
Results were averaged for the three trials at eadi of the lengths 4 and 
5, 6 and 7, 8 and 9 to arrive finally at the data labelled 4, 6 and 8 
in the sequel. (The middle digit was ignored for odd nui^ered lengths •) 
Full details are available in Keeton (1973). 

These aggregated repeated measures are reported in tables 3 and 4 for the 
two independent groups of pupils, one from an inner-city (lov; SES) school 
and one from a suburban (high SES) school. ^ Thus, tables 3 and 4 reveal 
the 2x2x3 design on the repeated measures and together illustrate the 
iimple one=way design on the. subjects. 

The h^othesis of primary interest v;as that the inner^ city children 
would score higher on the second half of the series than the suburban 
children and vice versa for the first half* In analysis of variance 
terminology, there would be a significant SES k Scoring interaction, A 
nmrtoer of other hypotheses made a complex analysis dasirable. 

The Transformation and the Data 

A 12 X 12 orthogonal matrix was generated Wiat refleGted the 2 k 2 x 3 
aesign on the u-epeated measures, Multivariance (Finn, 1971) contains 
a featured at makes ttiis a- single matter/ The data are entered .1 



the program as 12 different dependent variables, taking care that tliG 
order of entry corresponds to the. 2x2x3 design. In this exaniple, 
there are exactly 50 ehildren in each group, yielding two 12 x 1 vectors 
of means, Applyincj the 12 x 12 trans formation matrix yields 12 new 
variables for yach group corresponding . to the main effects and inter^ 
actions of a 2 x 2 x 3 analysis of varianco. The entries in the actual 
matrix are fractional quantities dictated by the requirement of ortho- 
gonality, but the pattern is a familiar one; sample rows are shown to 
one decimal place in table 5. The structure of the vector of means is 
shown for comparison so that the ineaning of the new "variables" can be 
seen. Because one of the factors, length, has ordered, equally- spaced 
levels, it suggests a model in which orthogonal polynomials are used 
to study the linear and quadratic components of the variability due to 
length. (In fact, such a model accounts for slightly more variance than 
one containing only simple difference contrasts, ) Use of orthogonal 
polynomial coefficients accounts for the presence of zeros in table 5. 

Results 

A complete sunmiary table is shown as table 6, Comparison with, table 1 
Bhrnm the same pattern of new variables within group effects/ the lov/er 
half representing all the. Group x (new variaiile) interactions* Both 
tests of mean vectors (Multivariate) show significance beyond .01, an 
almost guaranteed result for the constant term because of the grand mean 
itself. The significant test of groups as a whole shows that a fairly 
complex pattern exists in the data. 

Looking at the top half (constant) , we see all but two univariate tests 
significant beyond ,01. This cannot be taken at face value because as 
we noted, when we go beyond two repeated measures the new variables are 
not un correlated, A look at the cross-product terms shows a nuitiber 
of large entries relative to the siims of squares. We have to turn to 
the step-^down tests, a saries of step-wise regression results with all 
variables above a given one included in the regression equation. Note 
that the new variable scoring (P-R) is not even significant if we remove 
the effect of the grand meani This odd result is marked in left and 
right margins with e. 

Only Mode and Scoring x Mode survive the step-down analysis' in the top 
group. Looking below, however, we see several interactioii^ witii Mode^ 
so we will not pursue the main effect. Before going farther, however, 
let us acknowledge that the step-down tests are order sensitive and ask 
the readers to accept the authors' word that various orders were tried 
and the same robust effect emerged each time. Coi^liments are due 
Multivariance (1,®^ Finn) againi in that repeated analyses changing only 
the order of the dependent variables (or their mmtoer) are both easy 
and economical of computer time. This is possible because only the 
very last stage of the analysis need be redone, using most Galeulations 
ovfr again. : ' ■ = 



The affect of principal interest, SES x Scoring, is the first one beJow 
the line in tha bottom group, that is, G x (p-R) . it is significant 
both in univariate and stsp-down tests, no matter which variables are 
entered before (above) it. The hypothesis is thua strongly and validly 
supported, untainted by approximate F-taBts or other adhocery. An 
unusual reversal is marked by ® , where G x (V-A) appears significant 
in the step-down but not in univariate. This is apparently an artifact, 
sinoe it does not happen with otter' orders , 

As evidence of the truth of the maxim, "Be kind to your data and your 
data will be kind to you," wa' offer the seemingly unlikely three way 
interactions (marked by *) , G x (V-A) x L2 and G x S x Ll. Looking at 
the sums of squares it is clear that the pooled Ll and L2 effects would 
be sicnificant, so we plot the means in figure 1. 

The interesting G x S effect already mentionad turns out to be only part 
of the story. There is a difference in the slopes of the lines 
(G X S X Ll) , and B very interesting difference tooi The lo SES 
(inner-city) children show a remarkably different pattern in recenay 
from their primaoy scores as Umgth of series incraases, while the Hi SES 
(suburban) group is consistant. That finding led to a second study, 
being reported tomorrow afternoon (Keeton and McLean, 1973).. 

Mode of presentEtion (visual vs auditory) is confusing theoretically, 
confounded experimentally and hopeless staiistioally. The confounding 
occurrad because the children spon+-aneously rehaaraed the digits 
verbally in tha auditory presentation but not in the visual and because 
auditory .always followed visual. None of this explains why the suburban 
group did so poorly at the auditory task for series of length 8 and 9 
(see figure 1, G x (V-A) X L) . The graphs are plotted primarily to show 
that when a result appears artlfactual in the analysis, e.g. G x (V-A), 
its graph shows nothing, and when a result appaars robust, e.g. 
G X (V-A) x L, its graph reveals a likely source of effect. If we can't 
interpret the results, is that Darrell Bock's fault? 

Conclusions 



Given the avail^ili^ of a powerful tool such as Multivariance, it seems 
doubtful we should ever analyze repeated measures othei- than as multi- 
variata data. Among the extensions not mentioned is the possibility of 
using covariates-, not usually possible because we have only one covariate 
measure per person and (by definition) several repeated measures. The 
covariates can be applied to each of the new "variables", though tfiis 
takes some extra work. (The trans formatiom matrix has to be entered 
via cards as part of a super-matrix and a p x p identity matrix, where p is 
the number of covariates.) • - 



The careful student of Bock^Finn will notice that this method is 
applicable to a single group as well as to data from more complGK 
designs. The authors nm have= data from a second year of the study 
reported here, prnmising a future paper on non^orthogonal multivariate 
analysis of covariance of repeatGd measures after orthogonal transformation 
marinated and ser/ed with sour creain. 
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